LHC discovery potential of the lightest NMSSM Higgs in the hi^aiai-^Afi channel 
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We explore the potential of the Large Hadron Collider to observe the /ii— >-aiai— >-4/x signal from 
the lightest scalar Higgs boson {hi) decaying into the two lightest pseudoscalar Higgs bosons (ai), 
followed by their decays into four muons in the Next-to-Minimal Supersymmetric Standard Model 
(NMSSM). The signature under study applies to the region of the NMSSM parameter space in which 
ruai < 2mr, which has not been studied previously. In such a scenario, the suggested strategy of 
searching for a four-muon signal with the appropriate background suppression would provide a 
powerful method to discover the lightest CP-even and CP-odd NMSSM Higgs bosons hi and ai. 



INTRODUCTION 

The Next-to-Minimal Supersymmetrie Standard 
Model (NMSSM) [l|-[l3} extends the particle content of 
the Minimal Supersymmetric Standard Model (MSSM) 
by one singlet superfield. The NMSSM has several at- 
tractive features beyond the MSSM. First^he NMSSM 
elegantly solves the so-called fj, problem [ij]: the scale 
of the fi parameter is dynamically generated at the 
electroweak or SUSY scale when the singlet Higgs ac- 
quires a Vacuum Expectation Value (VEV). Second, the 
fine-tuning and little hierarchy problems of the NMSSM 
are greatly diminished compared to the MSSM fisj . 
In the NMSSM, the upper mass limit on the lightest 
CP-even Higgs boson is higher than in the MSSM, 
making it less constrained experimentally. Another 
attractive feature of the NMSSM is that the lightest 
CP-even Higgs can have a significant branching fraction 
for the new /ii— >-aiai decay {hi and ai are the lightest 
CP-even and CP-odd Higgs bosons, respectively). This 
weakens the LEP-H constraints on the allowed Higgs 
parameter space, as this new decay channel reduces 
the branching fractions of hi into the modes used in 
direct Higgs searches. In addition, there are interesting 
implications in the cosmological Dark Matter sector of 
the model due to the appearance of the fifth neutralino, 
the "singlino." It has been shown ^ that the NMSSM 
is consistent with the experimentally measured relic 
density, and the data provide important constraints on 
the allowed NMSSM parameters. 

The rich phenomenology offered by the NMSSM, stem- 
ming from the extension of the scalar sector, has been the 
focus of numerous studies 17 - 25[ . In Ref. [31 , the first 
attempt to establish a "no-lose" theorem for NMSSM was 
presented. This theorem states that the Large Hadron 
Collider (LHC) has the potential to discover at least one 
NMSSM Higgs boson in the conventional mode, assum- 
ing that Higgs-to-Higgs decay modes are not important. 



However, the fact that Higgs-to-Higgs decay modes can 
be important has been shown in aiialyses devoted to 
re-establishing the 'no-lose' theorem |19l425| in the case 
where the /ii— >aiai branching fractions are significant 
and ai is light. NMSSM scenarios with between the 
2t and 26-quark thresholds {2mT < niai < 2mh) have 
previously been considered, focusing on the 4t [25| chan- 
nels in Higgs-strahlung and Vector Boson Fusion, estab- 
lishing the NMSSM No-Lose Theorem at the LHC ^ 
for this fli mass region. Future analysis of the 4r chan- 
nel is likely to be technically challenging and can only be 
performed with large datasets (typical integrated lumi- 
nosity of 10-100 fb~^). A more recent study |26[ has fo- 
cused on the same region using the /ii— >aiai— ^2/i2T pro- 
cess, which has the benefit that the invariant mass of two 
muons forms a much narrower peak, improving the sen- 
sitivity of such an analysis in spite of the large reduction 
in signal yield due to small B{ai—¥fifi). Our findings in- 
dicate a substantially higher QCD multi-jet background 
contamination as compared to Ref. (26| , which may have 
a substantial effect of the sensitivity of such a search. 

In this paper, we explore the region in which the oi 
mass is below the 2r threshold {rriai < 2mr). For this 
case, which has not been studied previously, we explore 
the potential of the /ii— >-aiai— 7>4/x signature at the LHC. 
Unlike searches for the 4t signature, the invariant mass 
of each muon pair provides a direct estimate of and 
the 4^ invariant mass provides mh^ ■ Use of these kine- 
matic constraints leads to essentially zero background 
and therefore allows one to rely on direct gg and bb fusion 
for Higgs production instead of the subdominant vector 
boson fusion or associated production processes chosen 
in the 4t search [l3| to suppress QCD backgrounds. 

We demonstrate that the analysis of the 4/i mode has 
excellent sensitivity to hi production, can be performed 
with early LHC data, and requires little in terms of de- 
tector performance except reasonably robust muon track- 
ing and identification. To present a realistic analysis, we 
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FIG. 1: (a): Mass of the lightest {hi) and second-hghtest (/12) CP-even Higgses as a function of jin/X and A. The density of 
generated scenarios surviving constraints is shown in the blue color scale, and the red line represents the single-valued A ^ 1 
limit (mean of A < 0.01 scenarios), (b): Mass of the lightest CP-odd Higgs (ai) as a function of A„ and A. The color scale is 
the average mass in each bin, and filled circles are the scenarios with < 2m-r. The edge of the low iriai region follows a 
parabolic curve, (30 GeV)A2 - A, ~ 0. 



use parameters of the CMS detector at the LHC to de- 
sign event selection and to estimate background contri- 
butions. 

The rest of the paper is organized as follows. In Sec- 
tion II, we study the NMSSM parameter space in which 
< and discuss the phenomenology of the model. 
In Section III, we review constraints on the NMSSM pa- 
rameter space from existing data. In Section IV, we out- 
line the proposed analysis for the mode and evaluate 
its sensitivity. Final conclusions are presented in Sec- 
tion V. 



the superpotential, A and k, the corresponding soft SUSY 
breaking parameters A\ and A^,, the effective ^ param- 
eter fi = Xs, the gaugino mass parameters Mi, M2, and 
M3, the squark and slepton trilinear couplings At, At, 
and At, and the squark and slepton mass parameters 
Mfj^ and M/^. For simplicity, we assume universality 
within 3 generations for the last two parameters, leaving 
only 6 parameters for sfermion masses. 



Parameter Scan of the Low-ma^ Region of the 
NMSSM 



NMSSM PARAMETER SPACE 

In our study we consider the simplest version of the 
NMSSM [Ml, in which the /ii?ii?2 term of the MSSM 
superpotential is replaced by 



\SH1H2 + -S^ 



(1) 



making the superpotential scale-invariant. In general, 
there are five soft braking terms; in the "non-universal" 
case, 



ml. Hi 



miS^ + XAXH1H2S + -A^S-\ (2) 



In the above equations, capital letters with tildes denote 
superfields while symbols without tildes denote the scalar 
component of the respective superfield. 



Soft breaking parameters in Eq.(l2]), 



m 



H2 



and 

rrig, can be expressed in terms of Mz, the ratio of the 
doublet Higgs VEVs, tan/3, and fi ~ Xs (where s = (S), 
the VEV of the singlet Higgs field) through the three 
minimization equations of the Higgs potential. Assum- 
ing that the Higgs sector is CP-conserving, the NMSSM 
Higgs sector at the Electro- Weak (EW) scale is uniquely 
defined by 14 parameters: tan /?, the trilinear couplings in 



To find the parameter space for our region of inter- 
est, niai < 2tot-, we scan th e NMS SM parameters using 



the NMSSMTools package |27H29l |. applying all known 



phenomenological and experimental constraints except 
the following: the cosmological dark matter relic density 
measured by WMAP [s^], the direct pp^hi^aiai^Aii 
search by the Tevatron l3l|. the direct e+e~— 
hi^aiQi searches by LEP |32l. [ssf . the direct T^'7ai 
searches by CLEO [s^l and BaBar [s^, and limits set 
by rare B^K£'^£~ decays [s^. These important con- 
straints are explicitly studied in our region of interest in 
a later section. 

In our scan, we fix parameters entering the Higgs sector 
at loop-level to M1/M2/M3 = 150/300/1000 GeV, At = 
Ab ^ Ar ^ 2.5 TeV, and Mf^ = Mf^ = 1 TeV. We 
then sample the NMSSM model points uniformly in a 
six-dimensional space. The first four scan parameters 
are conventional, broad ranges over the probable values 
of /i. A, tan/3, and Ax: 

• 100 GeV < /!< 1000 GeV 

• < A < 1 

• 1.5 < tan/3 < 50 
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FIG. 2: (a): Non-singlet fraction 1 — sin^ of the lightest CP-odd Higgs boson (ai) as a function of vs'm2l3/s. (b): Singlet 
fraction cos^ 6s of the lightest CP-even Higgs boson (hi) as a function of /ik/A. (The red line is the mean of A < 0.01 scenarios.) 



• -1 TcV < Ax <5 TcV 

For the two remaining parameters, we identify two ad- 
ditional phenomenological variables that allow a more 
narrow selection of the region of interest and simplify the 
interpretation of our observations. A theoretical justifi- 
cation of these variables is discussed in the next section. 
The first of these two parameters, /ik/A ~ ks, is selected 
because of its correlation with the mass of the CP-even 
Higgs bosons, see Fig. [T{a). The corresponding range 
used in the scan 

• < iik/X < 120 GeV 



was chosen to include two equally sized but phenomeno- 
logically different sub-domains; in the lower one hi is 
light and /i2 is the SM-like Higgs, and in the upper one 
hi is the SM-hke Higgs. 

The final parameter and its scan range, 

• GeV < (30 GeV)A2 - yl„ < 3 GeV, 

are chosen to zoom into the region of low ai masses as 
illustrated in Fig. (Hb). 
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sin 2/3 
2s 



(4) 



In general, ai is light in the regions of the parameter 
space approaching either the Peccei-Quinn (PQ) sym- 
metry limit (k— >^0) or the R-symmetry (RS) limit (A^, 
^A^-O). In both limits, ai is a massless axion, a fact 
which directly follows from Eq.(|3]). It can be decom- 
posed in terms of the weak eigenstates Hui, Hdi, and Si 
as (see e.g.js^l): 



oi = cos^pA -I- sinOpSi, 



(5) 



where A = cos f3Hui + sin (3Hdi- In the PQ limit, the 
mixing parameters cos9p and sin^p arc 



cos 9p 



V sin 2(3 



^v^ sin^ 2^ -I- 4s2 



, sin Bp 



2s 



v/w2 sin^ 213 + 4:3"^ 
(6) 

In the RS limit, the same parameters are 



Higgs Sector Spectrum and Mixings 

The CP-even and CP-odd Higgs mass matrices, M-s 
and Mp, can be written as pTj : 

Msii = .g^w^sin/3^ -|-//tan/3(AA + Ks) 



M 



■^533 

-^513 
-^523 



522 — g^v'^ COS (3'^ + ficot l3{Ax + Ks) 

, ^ sin 2/3 , ^ ^ , 
XAx z — - + Ks{Ai^ + 4ks) 



2s 

= {X^ - /2)v^ sm2l3 ~ Xs{Ax + Ks) 
= Aw(2Ascos/3 - sin/3(AA + 2ks)) 
= Aw(2Assin/3 - cos/3(Aa + 2ks)) 



(3) 



cos Up = 



V sin 2/3 



sin^ 2^ + s2 



, sm tip 



^v^ sin^ 2/3 + s2 



(7) 

According to Ea. (|6l7p . the non-singlet component of 
ai is determined by the ratio z;sin2/3/s. Figure [2t^a) 
shows evident correlation of the non-singlet fraction 
1 — sin^ Qp oi the lightest pseudoscalar Higgs boson ai 
with V sin 2/3/s, corresponding primarily to the PQ limit, 
as can be deduced from the slope of the correlation. It 
also demonstrates that in the region of interest ai is 
nearly a pure singlet, 1 — sin 6*1, < 1%, and that the 
values of sin 2/3/s are always below 0.1. If we define 
e = ■(;sin2/3/s, then up to 0(e), 0{k^), and ©(A^), the 



4 



1.5 



--0.5 



-1.5, 



nig < 2111^^ 








; (a) 









I 



20 



40 60 80 

^k/X (GeV) 



00 120 



1600 
1400 z 
1200 I 
-1000 Q. 

(D 

- 800 |. 

■< 

- 600 5' 

ut 

-400 S 

3 

-200 



o 

=3 
T3 




t 



-1.5, 



ITIa < 2m^ 

















1 



'O 20 40 60 80 100 120 

^k/X (GeV) 



1400 
1200 

z 

c 

1000 I 
m 

-800 g- 

E/> 

-600 ^ 
5' 

-400 S 
u 

-200 



"q. 

o 
u 



1.5r 



0) 



o" 
> 



-1.5, 



rria < 2m- 






x«-[ 











^0 20 40 60 80 
^^K/X (GeV) 



100 120 



11600 
1400 2 
c 

1200 I 
-1000 a 
800 3 
-600 5- 
400 i 
H200 




FIG. 3: Reduced couplings of hi to up-type quarks (a), down-type quarks (b), and vector bosons (c) as a function of i^ik/X, 
with the requirement that rUa^ < 2m.T. (The red hne is a mean of A < 0.01 scenarios.) 



CP-even Higgs mass matrix can be re-written as 



Higgs Couplings and Decays 



As 

/ {Ax + sk) tan 13 
-(Ax + sk) 



-{Ax + sk) 
{Ax + sk) cotp 



K 



(8) 
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One can see that e also determines the mixing of singlet 
and non-singlet CP-even Higgs states. For small values 
of e and A^ , characterizing the parameter space relevant 
to our study, the mass of the singlet CP-even Higgs bo- 
son is determined by 2sk ~ 2/iK/A. This substantiates 
our earlier observation depicted in Fig. [TJa) and the rel- 
evance of the fXK/X parameter used in our scan (Eq^. 
Further, in the sub-domain fin/X < 60 GeV, hi is light 
with m/ii ~ 2/x/t/A and has a significant singlet compo- 
nent, particularly for smaller values of A (and Ax). In 
the upper sub-domain /xk/A > 60 GeV, hi becomes the 
SM-like Higgs with m/^ ~ 120 GeV while /i2 acquires a 
large singlet component and mass mh2 — 2/xAt/A. This is 
illustrated in Fig.[2](b) showing the singlet fraction of hi, 
the lightest CP-even Higgs boson. 

To derive m^j, we diagonalizc the A4p matrix (Eq.Ul). 
Keeping 0{k), ©(A^), and ©(e^) terms, one has 



(9) 



where C, = sin2/3//i. Because 2^k/X determines the 
mass of the predominantly singlet CP-even Higgs boson, 
2^K,/X > rrih-^ and is always fairly large. Therefore, oi 
is light if (CA^ — Ak) is low, motivating the choice of the 
empirical parameter (30 GeV)A^ — used in the scan. 
The range of this parameter selects a region with 
between and approximately 30 GeV, avoiding most of 
the theoretically inaccessible region in which ^A^ — < 
and therefore m^^ < 0, as shown in Fig. [IJb). 



The couplings of hi and ai to each other and to 
Standard Model particles are determined primarily by 
their singlet and non-singlet components. While the CP- 
odd ai is always nearly a pure singlet (see Fig. HJa)), 
the singlet fraction of hi is correlated with iik/X, but 
also depends on the smallness of A. As illustrated by 
Fig. Hljb), for small A, hi is nearly a pure singlet in the 
fin/X <60 GeV sub- region, while in the /xk/A > 60 GeV 
domain, hi has negligible singlet component and is essen- 
tially the SM Higgs. Figure [3] shows a strong suppression 
of reduced couplings of hi to up- and down-type quarks 
as well as vector bosons in the fj,K/X < 60 GeV domain. 
This suppression leads to a severe reduction in the pro- 
duction rates of hi at colliders, making this scenario chal- 
lenging for experimental exploration. Fortunately, as will 
be shown later, small A values in the low /xk/A region are 
excluded by cosmological observations. 




40 60 80 
]XK/X (GeV) 

FIG. 4: Branching fraction of hi^aiai in the A versus fin/X 
plane, with the requirement that < 2mT. 
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Branching fractions of hi are determined by relative 
strength of the hi couphngs to SM particles and the 
hittiUi coupling, which is specific to the NMSSM. Be- 
cause Oi has a high singlet fraction, the singlet content 
of hi is directly related to the strength of the hiaiUi 
coupling. If this were the only effect, B{hi^aiai) would 
have been close to 100% in the lower half of the /xk/A 
domain and negligible in the upper half. However, this 
coupling is also proportional to A (see Eq. [2]) , which cre- 
ates a competing effect as larger values of A smear the 
nearly perfect separation of singlet- and doublet-type hi 
in the lower and upper halves of the /ik/X domain. The 
overall result is illustrated in Fig. HI showing average 

^oifli) for NMSSM models with < 2mr as 

a function of fin/X and A. It is evident that the sup- 
pression of hi SM couplings for /ik/A < 60 GeV makes 

J-aiOi) substantial as long as A is not too small. 
For the upper part of the /ik/A domain, B{hi^aiai) is 
small except for large values of A where the hi singlet 
fraction is enhanced. 




FIG. 5: Branching fraction of ai-^fifi for generated models 
as a function of ma^. Tlie red line is a mean of all scenarios 
as a function of ma^. The threshold at 3.55 GeV/c^ is 2mr. 
For 2m^ < rUai < 3m^ (the grey box), the branching fraction 
would be nearly 100%. 

As the lightest Higgs boson, ai can only decay to 
SM particles, even though its coupling to SM particles 
is strongly suppressed due to its nearly singlet nature. 
One should also notice that ai couplings to down-type 
fcrmions are proportional to tan/3 while its couplings to 
up-type fcrmions arc suppressed as l/tan/3. Therefore, 
ai branching fractions follow the standard mass hierar- 
chy of open decay channels to down-type fcrmions. Fig- 
ure[5] shows the the branching fraction for ai^fifi as ob- 
tained using NMSSMTools package. For niai < 2tot-, the 
ai^fifi channel becomes significant, making an analysis 
in the four-muon mode viable for experimental searches. 

It is important to note that the NMSSMTools calcu- 
lation of S(ai^/i/i) shown in Fig. [5] does not include 
hadronization effects important in the region < 
1 GeV/c^, and therefore is not reliable. One should also 
notice that for 2m^ < rUai < Sttt-tt, i3(ai— >-^^) is ex- 
pected to be about 100% because qq and gg decays are 



prohibited by hadronization and spin effects, and because 
77 is small. Since the status of i3(ai— in NMSSM- 
Tools for rUai < 0-5 GeV/c^ is not well-established and 
requires further development, we present our results only 
for the rriai > 0.5 GeV region. We would like to notice 
that B{ai^fifj.) is model-dependent in general and can 
be somewhat different in, for example, the Little Higgs 
model [il] used by the D0 collaboration in Ref. (sij . 
Therefore, we present our results as limits on production 
cross-section times branching ratios for the 4/i signature 
as a model-independent limit for a given topology and 
signature under study. 

CURRENT EXPERIMENTAL CONSTRAINTS 

Existing experimental data restrict the NMSSM pa- 
rameter space for the scenario studied here. In the fol- 
lowing, we discuss experimental measurements relevant 
to this scenario and evaluate their impact in restricting 
the allowed parameter space for models with low ma^. 

Cosmological Constraints 

The lightest NMSSM neutralino is a candidate for Cold 
Dark Matter (CDM). The WMAP measurement of the 
CDM relic density therefore serves as an important con- 
straint on the allowed NMSSM parameter space. In our 
scan, we used the MicrOmegas package [39| linked to 
NMSSMTools to calculate riNMSSM^i^ and determine if 
a particular model is consistent with the experimental 
data. We considered a model to be consistent with the 
CDM measurement if ^Inmssm/i^ < 0.1131 + 2 x 0.0034, 
corresponding to the 95% upper limit obtained using the 
latest WMAP 5-year dataset The NMSSM neu- 

tralino relic density need not account for all CDM ob- 
served by WMAP, but it cannot exceed it. 

To illustrate the effect of the WMAP constraints. 
Fig El^a) shows the density of generated NMSSM mod- 
els in the A versus /xk/A plane under the constraint that 
TOai < 2™^. Models that were determined to be consis- 
tent with the WMAP data are shown in Fig. El^b). The 
comparison shows that the WMAP bound excludes the 
region of small ^k/X and A. In that region, the lightest 
neutralino is light and weakly interacts with SM parti- 
cles, suppressing the neutralino annihilation rate and en- 
hancing the neutralino relic density to unacceptably large 
values. Figures [Zla) and (b) make the same comparison 
but in the niai versus rrih^ plane. 

Constraints from Direct Searches at Colliders 

Several searches for hi^aiai have been performed at 
collider experiments, with the strongest impact on the 
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liK/X (GeV) [iK/X (GeV) 

FIG. 6: Sampled points with rUa < 2mT and experimental constraints successively applied in the A vs. /xk/A plane. The low 
energy e^e~ data (CLEO and BaBar) have essentially no impact on the allowed parameter space. Color scale is number density 
and filled points are 100 models (before application of experimental constraints). 




m,(GeV/c') m,(GeV/c') 



FIG. 7: Sampled points with nia < 2mr and experimental constraints successively applied similar to Fig. |6]but in the nia vs. 
rah plane. The low energy e+e~ data (CLEO and BaBar) have essentially no impact on the allowed parameter space. Color 
scale is number density and filled points are 100 models (before application of experimental constraints). 



allowed NMSSM models coming from LEP-II data [331 ]. 
Although the singlet component of hi at low [injX and A 
(and correspondingly low m/^) would severely suppress 
h\ production in e^e^— these extreme scenarios 
are excluded by the WMAP data. LEP limits [H] on 
NMSSM models are inferred from /ii— J^aioi, oi— pairs of 
charm, gluon, and r jets; ai^ji^ limits were not quoted. 
The LEP-II upper limit on e+e^— >/iiZ with hi^aiai 
excludes models that predict rrih-^ within the kinematic 
limits, 45 < m/i^ < 86 GeV/c^, and TOai in the region of 
significant detector efficiency, rriai > 2 GeV/c^. 

In addition to LEP data, there were direct searches for 



T— >7ai by CLEO and BaBar at low energy e+e" collid- 
ers 3, 41 1 . Neither of these searches significantly con- 
strain the NMSSM models with low 771^^ because oi has a 
high singlet component, and thus negligible 66a 1 coupling 
(see Fig.[2][a)), for all sampled parameter values. Because 
CLEO and BaBar results have negligible effect on the 
allowed parameter space. Figs, ^c) and Ulc) show com- 
bined LEP-|-CLEO-|-BaBar constraints, but the reader is 
reminded that only LEP constraints are relevant. 

The flat distribution of rare B^Ki+l" de- 

cays [i^l could potentially set a limit on the parameter 
space under study through the bound on B{B^Kai) x 
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FIG. 8: Contributions to the pp^hi production cross-section at y/s = 14 TeV as a function of ^k/X and A, from gg (a) and bb 
(b), with the requirement that < 2m-r. (The red line is a mean of A < 0.01 scenarios.) 



B{ai-^fifi). However, this limit does not actually bound 
the region of our interest because the coupling of ai to up- 
type and down-type quarks in 6— >-sai penguin diagrams 
is suppressed due the highly singlet nature of ai. For 
fa = tan 6'p V sin(2/3)/2, the quantity /a tan^ /? is above 
100 TeV in our scan while the charged Higgs mass is typ- 
ically above 200 GeV/c^. Thus, the bound set in Fig. 2 
of Ref. [1^ is not relevant to the parameter space of our 
study. 



The results of a search [431 for the NMSSM with a low 
mass fli at the Tevatron was recently published by the 
D0 collaboration in the channel hi—^-aiai^Afi. With no 
excess of data over the SM expectations, the paper quotes 
95% C.L. upper limits for the cross-section of this pro- 
cess. To interpret the D0 result in terms of constraints 
on allowed NMSSM models in our scan, we calculate the 
NLO production cross-section for pp-^hi in the NMSSM 
using the SM NLO calculations for gg~^HsM uM ^^'^ 
bb^HsM with QCD-improved (running) Yukawa cou- 
plings 145|, corrected for differences in coupling between 
the SM and the NMSSM using NMSSMTools: 



(^{gg^hi) = aigg^HsAi) 
= cr{gg~^HsM) 



r(fei^gff) 
T{HsM-^gg) 
Brihi^gg)r'°\hi) 



T{HsM^gg) 



SM, 



\ YbbHs 



(10) 



(11) 



where cr{gg^HsM) and T{HsM^gg) are calculated us- 
ing HIGLU nil, while B{hi-^gg), r*°*(/ii), and the ra- 
tio of Yukawa couplings Ybbhi/YbbHsM are obtained us- 
ing NMSSMTools. For /z/t/A < 60 GeV (non-SM hi 
lighter than 120 GeV/c^), the cross-section is strongly 
suppressed even compared to the SM for low m^i be- 
cause hi has a large singlet fraction and weakly couples 
to SM particles (see Fig. [3]). For larger fin/X, the lightest 
CP-even Higgs hi becomes the SM-like Higgs and has a 
small hi-^aitti branching fraction. 

The D0 paper ^ quotes 95% C.L. hmits on 



a{pp^hi) X B{hi^aiai^4:fi) for several choices of 
with rrihi = 100 GeV/c^. To determine if a particular 
model in our scan is excluded by these data, we linearly 
interpolate the published cross-section limits for values 



of 



between the points in [43[. To obtain the ex- 



perimental cross-section limits as a function of , we 
need to correct for variations in the experimental accep- 
tance. We obtain those limits by taking the analysis ac- 
ceptance to be linear as a function of mh^ "increasin g by 
--10% when ruh^ increases from 80 to 150 GeV/c^" |4a | 
and matching it to the full analysis acceptance given at 
TO/ii — 100 GeV/c^. We then calculate the production 
cross-section and branching fractions for the model points 
and compare them to the values we derive from (43| . Fig- 
ures Eljd) and [TJd) show the density of NMSSM models 
surviving WMAP, LEP and Tevatron constraints. Be- 
cause of the suppression in production rate at low fin/X 
and small B{hi^aiai) at high ^uk/A, the Tevatron search 
has only a limited impact on the allowed NMSSM param- 
eter space, mainly excluding models with high A. A sig- 
nificant improvement in Tevatron reach for the NMSSM 
would require a large increase in integrated luminosity, 
thus requiring the LHC to make a definitive discovery or 
exclusion of NMSSM models with low ■ 



A DEDICATED SEARCH FOR THE LOW-rua^ 
NMSSM AT THE LHC 

Since oi is only weakly coupled to SM particles, it can 
only be produced at the LHC via decays of the lightest 
scalar Higgs hi^aiui. The main characteristic of such 
signal at the LHC is two back-to-back (in cf>) di-muon 
pairs of spatially nearby muons. The reconstructed di- 
muon pairs should have invariant masses consistent with 
one another, and their invariant masses also serve as a di- 
rect measurement of TOqi • Additionally, the 4/i invariant 
mass distribution should have a narrow peak correspond- 
ing to the nihi mass. We use these striking features to 
design an analysis suitable for early LHC running. 
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The four-muon final state considered in this analysis 
has relatively low experimental backgrounds. Therefore, 
instead of using the Vector Boson Fusion (VBF) produc- 
tion process chosen in the proposed NMSSM searches 
targeting the > region (25j . we focus on the 

largest Higgs production modes at the LHC, gg^hi and 
bb-^hi. Wc calculate the NLO cross-section for pp—^hi 
for the NMSSM by rescaling the LHC SM NLO calcula- 
tions |4J, |45| to correct for differences in couplings be- 



tween the SM and NMSSM (Eqs. [TO] and [11]). Like the 
Tevatron case, the cross-section is strongly suppressed 
compared to the SM if hi has a large singlet fraction. 
Figure [5] shows the production cross-section for 14 TeV 
pp^hi+X as a function of fin/X. While this suppression 
makes the analysis challenging even at the LHC, the con- 
straints arising from the WMAP relic density measure- 
ment exclude models with very low values of A, so the 
allowed models have small but non-negligible production 
cross-section. 



Analysis Selections 

We use Pythia to generate signal event templates with 
nihi in the range from 70 to 140 GeV/c^ and in the 
range from 0.5 to 4 GcV/c^. Wc chose the CMS detector 
as a benchmark for modeling a realistic experimental en- 
vironment, with parameters described in Ref. [47| . The 
important parameters for this analysis are muon momen- 
tum resolution, the minimum muon momentum needed 
to reach the muon system, geometric acceptance, and the 
average muon reconstruction efficiencies. Because of the 
large number of reconstructed muons in the event, we 
take the trigger efficiency to be 100%. 

The analysis starts by requiring at least four muon 
candidates with transverse momentum px > 5 GcV/c 
and pseudorapidity < 2.4 to ensure high and reli- 



able reconstruction efficiency. Of the four muon candi- 
dates, at least one must have pr > 20 GeV/c to sup- 
press major backgrounds and to satisfy trigger require- 
ments. Each event is required to have at least two pos- 
itively charged and two negatively charged muon candi- 
dates. For the surviving events, we define quadruplets of 
candidates, pairing the candidates by charge and sorting 
them into two di-muon pairs by minimizing the quantity 
{^R{^l,,^l,Y -f Ai?(^fc, where Ai?^ = /^jf + 
Muon quadruplets for which Ai? > 0.5 in either of the 
pairs are discarded as inconsistent with the signal topol- 
ogy. Acceptance for the selections listed above is shown 
in Fig.[5]for several representative values of nih^ and TOqi ■ 

The requirement of four sufficiently energetic muons 
in the event dramatically reduces contributions of po- 
tential backgrounds for this analysis. After accep- 
tance selections, the dominant background is due to 
the QCD multijct production where muons originate 
from heavy-flavor resonances, heavy-flavor quark decays, 
or from n/K decays-in-flight. We use Pythia to esti- 
mate the QCD multijet background, and obtain approxi- 
mately 2.6 events/pb~^ (approximately half containing 
at least one decay- in- flight). Using CalcHEP [4^ to 
estimate pp^M + X electroweak backgrounds, we ob- 
tain 0.04 cvents/pb~^. Direct J/ij] production is found 
by Pythia to be completely negligible. Other SM back- 
grounds (top, W-f-jets) are negligible in the region of in- 
terest of this analysis. 

The backgrounds are further reduced by requiring the 
kinematics to be consistent with the expected signal 
siganture. We calculate the invariant mass of each of 
di-muon pair, mi2 and TO34, as well as the invariant mass 
of all four muons, denoted as 77x1234. Figure ITOTa') shows 
the invariant mass of the muon pairs passing all selections 
in signal events for two choices of rrih^ and . Fig- 
ure [TO](b) shows the distribution of mi234 for two bench- 
mark points. To focus on the region of interest, we re- 
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FIG. 10: (a): 
(b): Invariant 



Invariant mass of reconstructed muon pairs for rriai = 0.5 and 3 GeV/c'^ (in both cases irihi = 100 GeV/c^). 
mass of four reconstructed muons for ruhi = 80 and 120 GeV/c^ (in both cases ruai = 3.0 GeV/c^). 



quire mi234 > 60 GeV/c^, mi2, 77134 < 4 GeV/c^, which 
reduces the QCD background to 0.4 events/pb~^. 

To ensure the compatibihty of the measured invari- 
ant masses of the two di-muon pairs, one could require 
k"-i2 — "T-341 < 0.08 + 0.005 X (mi2 + TO34). Such cut 
would require the two pair masses to be consistent with 
each other and would take into account the widening of 
absolute resolution in the reconstructed di-muon mass as 
a function of mass. If applied, the only background that 
still may be not completely negligible is the QCD multi- 
jet production, for which we conservatively estimate the 
upper bound to be 0.02 events/pb~^. However, instead of 
applying this selection explicitly, a better approach would 
be to fit the data in the 3D space ('71,12,77134,7711234), 
taking into account kinematic properties of the signal 
events. This approach maximizes the signal acceptance 
and therefore the statistical power of the analysis. It is 
also convenient from an experimental standpoint as the 
background events are distributed smoothly over the 3D 
space, allowing a fit of the 3D distribution to estimate 
backgrounds directly from the data. A potential signal 
would appear as a concentration of events in a small re- 
gion of the space (a 3D peak). We use a binned likelihood 
defined as a function of parameters , rrih^ and effec- 
tive signal cross-section a x B{hi^aiai)B^{ai^fj.fj,) to 
fit the simulated data using either background-only or 
signal-plus-background templates. We estimate the sen- 
sitivity of this analysis and present it in terms of the 
95% C.L. exclusion levels for signal cross-section using a 
Baycsian technique. 

Our estimations show that for an early data search 
(£ ~ 100 pb~^), the backgrounds are negligibe. For an 
analysis with higher luminosity, one can restore the zero- 
background situation by adding an isolation requirement 
to one or both of the di-muon pairs in the event. Iso- 
lation can be defined by cither setting an upper bound 



TABLE I: Expected rate of background events per 100 pb 
of luminosity after selection. 



Selection 


4 leptons 


QCD muhi-jet 


Pt(mi) > 20 GeV/c and 
PT(M«)>5GeV/c;i = 2,3,4 


4.8 ±0.2 


267 ± 23 


17112, 17134 < 4 GeV/c^ 
mi234 > 60 GeV/c^ 
177112 - 771341 < 0.08 GeV/c^ 
+0.005 X (mi2 + 77134) 


0.024 ±0.012 
0.010 ±0.007 

000+° °°5 

U.UUU_o.ooO 


90± 13 
39 ±9 

00+^-^^ 
"•uu^o.oo 



on the sum of the transverse momenta of all tracks in a 
cone around the reconstructed direction of the di-muon 
pair, excluding two muon tracks, or by rejecting pairs 
with additional tracks above a certain threshold. For 
the analysis with C = 1 fb~^ of data, we required no 
charged tracks with momentum px > 1 GeV/c in the 
y/ (Arj)^ + (A0)2 < 0.3 cone around the direction of at 
least one of the two muon pairs. This requirement is 
96% efficient for signal and reduces QCD multijct back- 
ground, dominated by events with muons originating 
from heavy flavor jets, by a factor of 6-7. For high- 
luminosity datasets, isolation can be further tightened to 
increase background suppression with only a moderate 
loss in signal efficiency. 



LHC Reach for NMSSM hi^mai 

We have proposed an analysis that has a potential of 
discovering the NMSSM with early LHC data in sce- 
narios with low niai ■ We estimate its sensitivity by 



10 



Ir- 
0.9^ 
0.8^ 
0.7^ 
0.6 r 

OAr 

0.3 r 

0.2'r 

0.1 i- 



1, with WMAP, LEP, CLEO/BaBar limits applied 
excluded by Tevatron 
excluded by LHC 100 pb'^ (no isolation) 
excluded by LHC 1 fb ' (with isolation) 
remaining 



*»»■■ 




40 60 80 

liK/X (GeV) 



^-2.5- 
o ; 

CD 

1r 
0.5 7 



x^- xx"*-** 



(b) 



40 50 60 70 80 90 100 110 120 130 

m^, (GeV/c') 



3 

g 2.5 

^ 2 

t 

'i 1.5 

CO 

t 1 



(C) * 



iif 

X X*! 



. I . . . . -I . . . . I . . 

5 10 15 20 25 

a(14TeV pp^ h) (pb) 



30 



FIG. 11: Sampled models with rUai < 2mr and all experimental constraints applied, presented as a function of model parameters 
(a), Higgs masses (b), branching fraction and LHC cross-section (c). With only 100 pb^^, the LHC's reach extends beyond 
that of the Tevatron. 



calculating the 95% C.L. upper limit on the product 
a{pp—^h)B{hi'^aiai)B^{ai^fifi) a, where a is the anal- 
ysis acceptance, using a Bayesian technique. Because of 
the low background, an upper limit on the signal cor- 
responds to approximately three reconstructed events. 
This hmit is 0.0293 pb for C = 100 pb-\ and scales 
linearly with luminosity assuming that the number of 
observed background events is zero. In nearly all pseu- 
doexperiments, this limit is independent of ruh^ and 
because the effective signal region that dominates sig- 
nal significance is essentially background-free and the 
probability to observe an event is small. Note that the 
corresponding projection for C — 1 fb~^ includes the 
isolation cut, slightly reducing signal efficiency and cor- 
respondingly loosening the limit. The upper limit on 
a{pp—^hi)B(hi^aiai) is shown as a function of m/j^ and 
niai in Table im 

Figure [TT] presents the region of NMSSM parameter 
space excluded by the Tevatron and the region that the 
LHC would exclude with 100 pb~^ (without isolation) 
and 1 fb^^ (with isolation), assuming no observed sig- 
nal. The regions are presented in the A, iik/X plane 
(Fig. 11(a)), the m^j, rrihi plane (Fig. 11(b)), and the 
plane of /ii— >-aiai— >-4/i branching fraction versus LHC 
pp^hi cross-section (Fig. 11(c)). High /ii-singlet sce- 
narios (which have low production cross-section) and low 
/ii-singlet scenarios (which have low /ii— >aiai branching 
fractions) are accessible to the Tevatron and the LHC 
to different degrees, leading to a region in Fig. ITlTc) 
where high /ii -singlet scenarios are excluded by the Teva- 
tron while some low /ii-singlet models with the same 
LHC cross-section times branching fraction are not. The 
Tevatron exclusion region has a sharp border only when 
viewed as a function of the Tevatron cross-section. 

It is worth noting that quantitative background esti- 
mates performed in our analysis may indicate that the 
LHC reach for NMSSM models with TOqi > in the 
2fi2T channel are substantially weaker than suggested 



TABLE II: 95% C.L. upper limit on aipp-^hi) x 
B{hi^aiai^Afi) (fb) at the LHC with C = 100 pb"^ (no 
isolation) and jC — 1 fb~^ (with isolation). The limit tightens 
at high rrihi because of the increase in acceptance with m^j . 



ma, (GeV/c") 



0.5 


1.0 


2.0 


3.0 


4.0 


Biai^Uli) (%) 


0" 


9.8 


15.2 


16.2 


0.7 


for £ = 100 pb ^ (no isolation) 


nih, = 80 GeV/c^ 
rrih, = 100 GeV/c^ 
rrih, = 120 GeV/c^ 


96.0 
74.8 
63.9 


110.3 
90.3 
77.4 


121.1 
100.8 
86.0 


122.6 
102.4 
90.8 


126.1 
103.9 
94.4 


for £ = 1000 pb"^ (with isolation) 


rrih, = 80 GeV/c^ 
ruh, = 100 GeV/c^ 
m^i = 120 GeV/c^ 


10.0 
7.8 
6.7 


11.5 
9.4 
8.1 


12.6 
10.5 
9.0 


12.8 
10.7 
9.5 


13.1 
10.8 
9.8 



"Recall that >aiai) is obatincd using NMSSMTools and is 

not reliable for rUa-, < 1 GeV/c^. Furthermore, this branching 
fraction is expected to reach nearly 100% for 2m,^ < ma, < 3m,-. 



in Rcf. [2^, which relied on extrapolating QCD back- 
grounds to avoid high-statistics simulations. Though 
the 4/i and 2/i2T analyses apply different selections, a 
rough extrapolation of the simulated QCD multijet back- 
grounds to the 4^ channel yields an estimate of back- 
grounds to the 2/z2t channel that is three orders of mag- 
nitude larger than what was used in Ref. [2^ , even with- 
out considering the much larger misidentification rate 
of hadronically decaying taus. The expected number 
of QCD multijet events in the TO1234 > 60, mi2 and 
TO34 < 4 GeV/c^ region of the 4fj, analysis, which is 
390 ± 90 evcnts/fb^^ (note that the numbers in Table U 
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are for 100 pb~^), could be reduced by a factor of 10-20 
using tight isolation requirements. Unlike our analysis, 
the study in Ref. [2^ applies restrictions on the trans- 
verse momentum of the di-muon pair, pf^'^ > 40 GeV/c, 
and on the missing transverse energy > 30 GeV. How- 
ever, the pt^'^ selection is similar to 7711234 > 60 GeV/c^ 
and, given typical expected resolution for multi-jet 
events, a > 30 GeV requirement cannot be powerful 
enough to overcome several orders of magnitude in esti- 
mated event count. Scaling the background estimate 
down by a factor of 10 to account for muon isolation, we 
expect about 39 ±9 cvcnts/fb^^ from QCD multijets, as 
opposed to the 0.03 events/fb~^ in Rcf. Allowing 
for the larger rate of tau misidentification compared to 
muons only increases the discrepancy. Another argument 
can be made using the D0 measurement in the 2//2t 
channel 43| , which had 1-2 expected background events 
in the narrow (±0.3-1.0 GeV/c^) windows around the se- 
lected points in the di-muon mass. Using common scal- 
ing estimates of background contributions from the Teva- 
tron to the LHC, one would expect similar backgrounds 
for LHC datascts of the order of 100 pb~^. For datasets 
with integrated luminosities of the order of 500 pb~^ , the 
corresponding QCD contamination would be 5-10 events 
per window. These much larger background estimates 
severely affect acievable exclusion limits and, consider- 
ing trial factors, would make any discovery in the 2/i2T 
channel with early data extremely challenging. 



CONCLUSIONS 

We have studied the phenomenology of the NMSSM 
scenarios with the mass of the lowest CP-odd Higgs bo- 
son, oi, below the 2r threshold. Our analysis of the 
impact of existing data on these models has shown that 
the WMAP and LEP-II data provide the most constrain- 
ing power, while recent CLEO and BaBar measurements 
have essentially no impact on the allowed parameter 
space, and the Tevatron data has only a weak impact 
on the allowed parameter space. As a result, a large 
fraction of the parameter space is not excluded by any 
existing data. We conclude that a new analysis should be 
performed at the LHC to definitively confirm or exclude 
these models. We propose an analysis suitable for the 
LHC using the 4/x signature, which has very low back- 
grounds and striking kinematical features allowing direct 
and precise measurement of the masses of the ai and hi 
bosons. Using the CMS experiment as a benchmark, we 
we estimate the sensitvity of such an analysis and demon- 
strate that it has the potential to either make a discovery 
or significantly diminish the allowed parameter space of 
the NMSSM with low rua^ using only 100-1000 pb'^ of 
early LHC data. 
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